Facial expression transfer is the act of adapting the facial expressions of a subject, such as an actor or home computer user, to a computer-generated (CG) target character. Mastering facial expression transfer and other aspects of facial animation is a long-standing challenge in computer graphics. The face can describe the emotions of a character, convey their state of mind, and hint at their future actions. Audiences are particularly trained to look at faces and identify these subtle characteristics. Accurately capturing the shape and motion of real human faces in the expression transfer process plays an important role in transferring subtle facial expressions of the subject to the CG character giving the CG character natural, life-like expressions.
Facial motion capture, a fundamental aspect of the facial expression transfer process, has come a long way from the original marker-based tracking approaches. Modern performance capture techniques can deliver extremely high-resolution facial geometry with very high fidelity motion information. In recent years the growing trend has been to capture faces in real-time, opening up new applications in immersive computer games, social media and real-time preview for visual effects. These methods approximate the three-dimensional (3D) shape and motion of a face during the performance using either depth or web cameras. To make this tractable, many real-time approaches use generic, coarse-resolution face models (e.g., numerical meshes) as a basis for the reconstruction.
While coarse-resolution models simplify the capture problem and facilitate transferring facial expressions to the animated character, they may not capture the unique finer-scale facial details of the individual performer, such as wrinkles on the forehead or the so-called crow's feet around the eyes. As a result, real-time performance may come at the cost of facial fidelity and there exists a gap in reconstruction quality between current offline and online capture methods.
Embodiments of the disclosure address these and other challenges.